Commonly, interactive voice response systems (IVR) are created through a combined use of software and human input. One technique is for a business to set up a human operated system disguised as an automated system to handle interactions with callers. The business then observes how callers interact with the human operated system, and builds an actual automated system based on those observations. Such a technique is generally referred to as using a “Wizard of Oz” system, because a human input is hidden from callers behind a façade of automation.
Current Wizard of Oz systems have many drawbacks which decrease their overall efficiency. For example, a business is forced to build the human operated system from the ground up in order to begin the process. Additionally, current Wizard of Oz systems deploy the human and automated systems in sequence. Therefore, it is not possible to directly compare the decisions made by human systems with decisions made by the automated systems being designed. Further, it can often be difficult to effectively sort through the data created during caller interactions with the human system to know what interactions will be most problematic for the automated system and how those interactions should be dealt with. The teachings of this application can be used to help address one or more of those problems with present Wizard of Oz technology, as well as providing other benefits which are apparent in light of the disclosure contained herein.